Biotic and abiotic controls on microbial activity, functionality and processes in soil hotspots

When a carbon (C) source is added to a soil volume and causes microorganisms to thrive, a microbial hotspot is formed — a microsite with much higher process rates than the surrounding bulk soil. The formation, size, and lifetime of microbial hotspots in the vicinity of growing roots (rhizosphere) are governed by biotic and abiotic factors. Root exudation stimulates the production of extracellular enzymes and thus soil organic matter (SOM) decomposition. Nonetheless, we lack a mechanistic understanding of the enzymatic response of rhizosphere microorganisms to individual component of the root exudates. The extent and spatial distribution of rhizosphere depend on soil matrix (e.g. soil nutrient availability), plant properties (e.g. root morphology) and climate changes (e.g. warming and drought). The ongoing global change and the manipulation of exudate composition or root morphology often occur simutaneously, but the consequences of their interactive effects on microbial processes are poorly understood. For example, how the presence of root hairs regulates exudate input and microbial strategies in response to climate changes remains unknown. Therefore, this thesis coupled novel methods including high-throughout sequencing and in situ imaging approaches to demonstrate the effects of biotic or abiotic factors and their interactions on microbial localization, community structure, activity, strategy and efficiency. The objectives of this thesis are to investigate: 1) which component of root exudates plays the key role in stimulating microbial enzyme activities in the absence of living roots and how the presence of root hairs or the presence or absence of inhibitory substances within root exudates influences the formation of rhizosphere gradients; 2) the effect of soil nutrient availability on kinetic parameters in hotspots and bulk soil; 3) whether the response of enzyme activities to drought is driven by the selection of drought-resistant microorganisms; 4) how interactions of root hairs and warming or interactions of root hairs and drought affect the spatial distribution of enzyme activities and unravel strategies for plants and microorganisms to adapt to climate changes. Using artificial roots in combination with specific exudate components, we identified that the spatial distribution of microbial-derived enzyme activities is enzyme- and component-specific: 1) alanine had the overall strongest effect in this nitrogen (N)-limited soil; 2) the activities of phosphorus (P)-, N- and sulfur (S)-related enzymes showed clear gradients in the rhizosphere while the pattern for enzymes majorly involved in C-cycling was uniform and independent of the exudate composition. We also found benzoxazinoids presented in exudates suppressed β-glucosidase activity by 30% (mutant with reduced benzoxazinoid content in exudates (bx1) vs. its corresponding wild type maize), but the presence of root hairs increased exudate release and expanded the spatial extent of β-glucosidase activity around the root axis by 35% (mutant with defective root hair prolongation (rth3) vs. its corresponding wild type maize). Apart from biotic factors, abiotic factors have fundamental effects on microbial processes and microbial community structure. Effects of soil C and nutrients status on functional properties of microorganisms in soil hotspots were investigated by coupling zymography and measurements of kinetics of substrate-induced growth response and enzyme activities in two soils with contrasting soil nutrient availability. The result showed that: 1) differences in microbial growth strategy between rhizosphere hotspots and bulk soil were dependent on soil nutrient availability; 2) differences in enzyme activity and affinity were detected between hotspots and bulk soil in both soils but were enzyme-specific: the difference was significant for β-glucosidase, whereas it was insignificant for leucine aminopeptidase. Drought (another abiotic factor) only induced minor changes in bacterial community structure in rhizosphere hotspots, instead, it increased relative abundance of genera belonging to Actinobacteria capable of leucine aminopeptidase and chitinase production and thus induced a 5.0 - 17% increase in the number of gene copies encoded by Actinobacteria related to these two enzymes. This was reflected in a 35 - 70% increase in the activities of leucine aminopeptidase and chitinase under drought. This demonstrates that bacterial communities react to drought stress by increasing extracellular enzyme production and they achieve this by encoding more enzyme - genes. Root hairs proliferation and warming strongly influence exudate release, enzyme activities and microbial substrate utilization. To reveal their interactions, mutant with defective root hair prolongation (rth3) and its corresponding wild type maize were grown for 3 weeks at 20 and 30 ℃, respectively. Root hairs regulated enzyme expression, microbial growth strategies and thus substrate use efficiency, hence mediating the SOM stocks in response to warming. To clarify the relative importance of biotic and abiotic factors, we conducted an experiment considering the interactive effects of root hairs and drought. The result showed that both root hairs and soil moisture influenced spatial distribution of rhizosphere biochemical properties and processes, but soil water availability was more important than root exudates when the soil is limited by both water and C sources. In summary, both biotic and abiotic factors could influence microbial processes in hotspots. The role of root hairs was diminished when interacting with drought even though it was of great importance in regulating enzyme systems and microbial growth to adapt to climate warming. Therefore, we suggest that predicting and modeling soil C and nutrient dynamics should not only incorporate the interactive effects of biotic and abiotic factors, but also consider which factor involved in the interaction. Overall, this thesis strongly contributes to the understanding of mechanisms involved in the hotspot’s processes. This is particularly crucial for agro-ecosystems where many essential ecosystem services relevant to human well-being are linked to micro-scale processes occurring in hotspots.2021-08-2

Rethinking the design and implementation of the i/o software stack for high-performance computing

Current I/O stack for high-performance computing is composed of multiple software layers in order to hide users from complexity of I/O performance optimization. However, the design and implementation of a specific layer is usually carried out separately with limited consideration of its impact on other layers, which could result in suboptimal I/O performance because data access locality is weakened, if not lost, on hard disk, a widely used storage medium in high-end storage systems. In this dissertation, we experimentally demonstrated such issues in four different layers, including operating system process management layer and MPI-IO middleware layer on compute server side, and parallel file system layer and disk I/O scheduling layer on data server side. This dissertation makes four contributions towards solving each of the issues. First, we propose a data-driven execution model for DualPar to explore opportunity of effective I/O scheduling to alleviate I/O bottleneck via cooperation between the I/O and process schedulers. Its novelty is on the ability to obtain a pool of pre-sorted requests to I/O scheduler in its data-driven execution mode by using process pre-execution and prefetching techniques. Second, realizing that well-formed locality for an MPI program by using collective I/O can be seriously compromised by non-determinism in process scheduling, we proposed Resonant I/O, to match the data request pattern with the pattern of file striping over multiple data servers to improve disk efficiency. Third, since the conventional practice for I/O parallelism using file striping may compromise on-disk data access locality, we proposed IOrchestrator scheduling framework which is implemented in PVFS2 parallel file system to improve I/O performance of multi-node storage systems by orchestrating I/O services among programs when such inter-data-server coordination is dynamically determined to be cost effective. Fourth, we developed iTransformer, a scheme that employs a small SSD to schedule requests for the data on disk. Being less space constrained than with more expensive DRAM, iTransformer can buffer larger amounts of dirty data before writing it back to the disk, or prefetch a larger volume of data in a batch into the SSD. In both cases high disk efficiency can be maintained for highly concurrent requests

Orthrus: A Framework for Implementing Efficient Collective I/O in Multi-core Clusters

Abstract. Optimization of access patterns using collective I/O imposes the overhead of exchanging data between processes. In a multi-core-based cluster the costs of inter-node and intra-node data communication are vastly different, and heterogeneity in the efficiency of data exchange poses both a challenge and an opportunity for implementing efficient collective I/O. The opportunity is to effectively exploit fast intra-node communication. We propose to improve communication locality for greater data exchange efficiency. However, such an effort is at odds with improving access locality for I/O efficiency, which can also be critical to collective-I/O performance. To address this issue we propose a framework, Orthrus, that can accommodate multiple collective-I/O implementations, each optimized for some performance aspects, and dynamically select the best performing one accordingly to current workload and system patterns. We have implemented Orthrus in the ROMIO library. Our experimental results with representative MPI-IO benchmarks on both a small dedicated cluster and a large production HPC system show that Orthrus can significantly improve collective I/O performance under various workloads and system scenarios.

Margin Maximization in Attention Mechanism

Attention mechanism is a central component of the transformer architecture which led to the phenomenal success of large language models. However, the theoretical principles underlying the attention mechanism are poorly understood, especially its nonconvex optimization dynamics. In this work, we explore the seminal softmax-attention model $f(\boldsymbol{X})=\langle \boldsymbol{Xv}, \texttt{softmax}(\boldsymbol{XWp})\rangle$, where, $\boldsymbol{X}$ is the token sequence and $(\boldsymbol{v},\boldsymbol{W},\boldsymbol{p})$ are tunable parameters. We prove that running gradient descent on $\boldsymbol{p}$, or equivalently $\boldsymbol{W}$, converges in direction to a max-margin solution that separates $\textit{locally-optimal}$ tokens from non-optimal ones. This clearly formalizes attention as a token separation mechanism. Remarkably, our results are applicable to general data and precisely characterize $\textit{optimality}$ of tokens in terms of the value embeddings $\boldsymbol{Xv}$ and problem geometry. We also provide a broader regularization path analysis that establishes the margin maximizing nature of attention even for nonlinear prediction heads. When optimizing $\boldsymbol{v}$ and $\boldsymbol{p}$ simultaneously with logistic loss, we identify conditions under which the regularization paths directionally converge to their respective hard-margin SVM solutions where $\boldsymbol{v}$ separates the input features based on their labels. Interestingly, the SVM formulation of $\boldsymbol{p}$ is influenced by the support vector geometry of $\boldsymbol{v}$. Finally, we verify our theoretical findings via numerical experiments and provide insights

Zero-shot stance detection based on cross-domain feature enhancement by contrastive learning

Zero-shot stance detection is challenging because it requires detecting the stance of previously unseen targets in the inference phase. The ability to learn transferable target-invariant features is critical for zero-shot stance detection. In this work, we propose a stance detection approach that can efficiently adapt to unseen targets, the core of which is to capture target-invariant syntactic expression patterns as transferable knowledge. Specifically, we first augment the data by masking the topic words of sentences, and then feed the augmented data to an unsupervised contrastive learning module to capture transferable features. Then, to fit a specific target, we encode the raw texts as target-specific features. Finally, we adopt an attention mechanism, which combines syntactic expression patterns with target-specific features to obtain enhanced features for predicting previously unseen targets. Experiments demonstrate that our model outperforms competitive baselines on four benchmark datasets

FedYolo: Augmenting Federated Learning with Pretrained Transformers

The growth and diversity of machine learning applications motivate a rethinking of learning with mobile and edge devices. How can we address diverse client goals and learn with scarce heterogeneous data? While federated learning aims to address these issues, it has challenges hindering a unified solution. Large transformer models have been shown to work across a variety of tasks achieving remarkable few-shot adaptation. This raises the question: Can clients use a single general-purpose model, rather than custom models for each task, while obeying device and network constraints? In this work, we investigate pretrained transformers (PTF) to achieve these on-device learning goals and thoroughly explore the roles of model size and modularity, where the latter refers to adaptation through modules such as prompts or adapters. Focusing on federated learning, we demonstrate that: (1) Larger scale shrinks the accuracy gaps between alternative approaches and improves heterogeneity robustness. Scale allows clients to run more local SGD epochs which can significantly reduce the number of communication rounds. At the extreme, clients can achieve respectable accuracy locally highlighting the potential of fully-local learning. (2) Modularity, by design, enables $>$100$\times$ less communication in bits. Surprisingly, it also boosts the generalization capability of local adaptation methods and the robustness of smaller PTFs. Finally, it enables clients to solve multiple unrelated tasks simultaneously using a single PTF, whereas full updates are prone to catastrophic forgetting. These insights on scale and modularity motivate a new federated learning approach we call "You Only Load Once" (FedYolo): The clients load a full PTF model once and all future updates are accomplished through communication-efficient modules with limited catastrophic-forgetting, where each task is assigned to its own module.Comment: 20 pages, 18 figure